Apparatus and method for preparing, storing, transmitting and displaying images

ABSTRACT

An imaging and display apparatus for passive displays evaluates the illumination of an input scene and incorporates data representative of such input scene within a transfer media. The transfer media may be a broadcast or transmission of image data, illumination data and gamma information that can be received by a display system, which includes a passive display illuminated by incident light, to display images while adjusting the incident light and/or gamma based on the received data to reduce energy requirements, to increase contrast or shades of gray in the displayed image, and to optimize light source operation for color fidelity. The data may be provided as a video signal, modulated video signal, s-video signal, digital signal, or other signal that can be used by a passive display system to display images.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority of U.S. Provisional Patent ApplicationSer. No. 60/453,147, filed Mar. 10, 2003, which is incorporated byreference.

TECHNICAL FIELD

The present invention relates generally to apparatus and method for usein the field of display, and, more particularly, to apparatus and methodin which illumination information and/or other information may beobtained and stored or transmitted in addition to image data. Theinvention also relates to apparatus and method for one or more ofpreparing, storing, transmitting and displaying images.

BACKGROUND

In the course of preparing images for display, such as, for example,movies, whether of the amateur or home type or professional type,videos, and the like, a camera typically would record one or more imagesof one or more scenes. The image information pertaining to therespective scene(s) may be processed and provided as image data.Processing may include, for example, arranging the data in a particularformat for recording or for broadcasting. Various formats in which suchdata is stored and broadcast are known and new formats may be developedin the future. The image information and, thus, the image data mayinclude, for example, respective R, G, B values, Y, U, V values,intensity, hue, saturation, and/or other information that may be used indisplaying the image on a television, monitor or some other display. Anexample of a signal carrying such information is a video signal; anotherexample is a digital signal. The present invention is not limited to theparticular signal type or to the format of such signals.

One example of a format for image data storage is that in which the datarepresenting the image characteristics of a frame, such as a momentaryimage of a scene, whether a “real” scene or an animated scene, may bestored in a storage medium, such as a DVD, CD, tape, computer harddrive, or some other memory; a number of sequential frames provide for asequence of displayed images to display a motion picture, video, etc.One approach for storing image date for television display uses twofields to compose a frame, the image data for respective fields beingstored in odd or even lines corresponding to the horizontal scan linesof a conventional television CRT (cathode ray tube) type of display. Thepresent invention, as is described in detail below, is concerned withimage data and is not limited to the format in which that image data isstored, such as, the number of fields from one to any larger number offields per frame.

In the course of filming scenes, by using video camera, digital camera,film camera, or any other means to obtain images, illumination levelordinarily is determined by the amount of light incident on, reflectedfrom or produced by objects in the scene or otherwise directed forpickup by the camera. The brightness characteristics of imageinformation from a given scene as recorded by a camera may be adjustedby adjusting the camera aperture and/or shutter speed. Also, an opticalfilter may be used to change the appearance of a scene to a camera and,thus, characteristics of the image information obtained or recorded bythe camera. However, there are a number of limitations on such filmingtechniques. For example, it may be rather inconvenient and timeconsuming to change optical filters during the course of filming ascene. Also, as illumination levels change, the effective sensitivity ofthe camera may be changed, whereby the amount of data perceivedaccurately by the camera may be diminished, e.g., loss of resolution,contrast, etc. Thus, there is a need to improve such resolution andcontrast.

There are a number of different types of displays able to displayimages, such as movies, videos, still images, and the like. One type ofdisplay is a passive display. A passive display usually operates bymodulating light that is provided thereto (incident light). An exampleof a passive display is a liquid crystal display, and another example ofa passive display is known as a digital micro mirror device (DMD), suchas that sold by Texas Instruments Incorporated. There are a number ofliquid crystal display devices, such as, for example, those known astwisted nematic, supertwist, polymer disbursed liquid crystal (PDLC)also known as encap (NCAP), and ferroelectric; these are examples, andothers may exist now or in the future. Another type of display is anactive or light emitting display, which provides light output withoutthe need for a separate illumination source or light source; examplesinclude cathode ray tubes (CRT), electroluminescent displays (EL),plasma displays, and others that may exist now or in the future.

The displaying of a dark scene using a passive display encounters adisadvantage that ordinarily is not present for active displays. Theproblem has to do with reduced resolution and/or contrast of thedisplayed dark image. In an active display, such as a CRT, for example,when it is desired to display a dark scene, the intensity of the outputlight can be reduced by reducing input to individual pixels. The partsof the dark scene may be output at the reduced brightness orillumination level. However, the number of light producing pixels doesnot have to be reduced; all pixels of the CRT can be active so thatresolution is maintained even though intensity of the light produced bythe pixel phosphors, for example, may be reduced. A pixel sometimes isreferred to as a picture element or pel, a phosphor dot in a monochromedisplay, especially a CRT type, or a group of three (red, green andblue) phosphor dots for a multicolor display, etc.

However, in a passive display, such as a liquid crystal display, a priorapproach to reduce brightness of a displayed image or scene has been toreduce the number of pixels which are reflecting or transmitting lightat a particular moment to form a relatively dark image. Such a reductionreduces the resolution and/or contrast of the display. Such a reductionalso may adversely affect gamma characteristic of the display and/or ofparticular images provided by the display.

The pixels may be discrete pixels or may be blocks or areas where anoptical signal or optical output can be developed by emission of anactive display or by reflection or transmission of a passive display.The optical signal referred to may mean that light is “on” or providedas an output from the device or that the pixel has its other conditionfor not producing or providing a light output, e.g., “off”; and theoptical signal also may be various brightnesses of light or shades ofgray. The optical output or optical signal produced by a pixel may be acolor or light of a particular color. These and other operatingcharacteristics of displays are known and quite standard in the field ofdisplay technology.

The human eye has difficulty distinguishing between seeing orrecognizing the difference between low and high brightness and contrastranges. This difficulty is increased when the number of pixels isdecreased and resolution is degraded. An approach to improve resolutionand contrast in a passive display is described in U.S. Pat. No.5,717,422, which is incorporated in its entirety by this reference.Other pending patent applications that describe approaches to increaseresolution and contrast for a passive display, even when the display isshowing a relatively dark image, are described in U.S. Pat. No.6,184,969, issued Feb. 6, 2001, and co-pending U.S. patent applicationSer. No. 09/676,915, filed Oct. 2, 2000, the entire disclosures of whichare incorporated by this reference. The patents and patent applicationjust mentioned describe controlling the intensity of light supplied to alight modulating passive display as a function of a brightnesscharacteristic of the image being displayed.

In the '422 patent is disclosed a passive display apparatus, such as anLCD, and method for displaying images with high contrast by controllingthe light input to the display to control brightness of the output imagewhile operating respective pixels of the display to obtain goodresolution and contrast without regard to the output brightness.Different color effects also can be obtained. As is described in the'422 patent, an image of a candle lit room would be relatively dim. Theprior art passive displays would use a relatively small number of pixelsto provide light that creates the image, whereas a relatively largenumber of pixels would be used to block light to give the effect ofreduced intensity or dim room. In the invention of '422 patent, though,the number of pixels used to create the imagery does not have to bereduced to reduce light intensity or brightness of the image; rather,the intensity of the illuminating or incident light changes to diminishthe brightness of the image. Therefore, image data would not be lost asbrightness of an image is decreased. Thus, the amount of informationthat can be conveyed by the display in creating the image is increasedover the capabilities in the prior art.

As an example of increased information provided by the invention of the'422 patent, one could obtain a gray scale of 100 shades of gray byusing a passive display that provides 10 shades of gray and anilluminating source that provides light at 10 different levels;multiplying the display capability times the illuminating sourcecapability yields 100 shades of gray. Gray scale capability can beincreased further using a field sequential color display in which theilluminating light is provided sequentially as red, green, and bluelight, each of which can be modulated separately by the display. Widerange of gray scale is advantageous in head mounted displays, e.g.,virtual reality displays or other head mounted displays, where immersionin the image is desirable. Using features of the '422 patent, as werejust described, high illumination can be provided a scene as it isfilmed, yet the gray scale and contrast ratio of the image as actuallydisplayed can be adjusted by adjusting the illuminating source for thedisplay without loss of image data or with minimal loss of image data.Thus, a high contrast image can be presented. Also, adjustments can bemade selectively to alter images so that, for example, a sunrise scenecan be provided in which red portions of the image are enhanced and blueand green are minimized.

By separating the two functions of brightness (according to theintensity of the illuminating source) and image (based on operation of apassive display, e.g., a liquid crystal modulator), images can beadjusted to achieve a desired result. An example is to photograph ascene in daylight to get good resolution and contrast, and then byadjusting the illuminating source and/or the colors of the illuminatingsource, the impression of a moonlit scene, a candle lit environment,sunrise or sunset, etc., can be obtained.

Gamma is a characteristic or parameter that is used in the field ofdisplay technology. The Adobe Photoshop Version 7.0 software describesgamma in relation to the brightness of midtone values. As describedthere, the midtone values from black to white as produced by a monitorare nonlinear and, therefore, would be represented graphically as acurve rather than as a straight line. The slope of the curve halfwaybetween black and white is what is defined by that software as the gammavalue. Such software provides the possibility of adjusting gamma toimprove the accuracy of a displayed image in reproducing the actualcolors of a scene that is represented by the image. According to anotherconsistent definition, gamma is the transfer function from the inputlight to the output image.

Gamma correction in the field of computer graphics also concerns controlof overall brightness of an image. Gamma correction is desirable toobtain accurate displaying of images on a computer display or otherdisplay, display system, monitor or television. Note that the termsdisplay, monitor, television and the like are used synonymously hereinunless otherwise expressed or indicated by context. Varying the amountof gamma correction that is applied in a given display system may changebrightness and also the ratios of red, green and blue colors, forexample, that are displayed. Gamma correction is provided inconventional display systems in various ways to take into account thatthe intensity of a given pixel may have a non-linear relation to thedrive signal, e.g., the drive voltage, for that pixel. As one example,for a conventional cathode ray tube monitor, the intensity to voltageresponse curve may be on the order of a 2.5 power function. Therefore,if for a given pixel an intensity voltage representing an intensity of iwere to be delivered to the monitor, the monitor actually would provideintensity of i^(Λ)2.5 (i to the 2.5 power). Therefore, the actualvoltage supplied to the monitor must be corrected, i.e., gammacorrected, so the proper intensity is displayed. As is evident, gammacorrection can be relatively complicated and in many instances the userof a display is not provided the ability manually to adjust gamma.

Color fidelity is the extent of accurate representation of the colorcharacteristics of a scene as portrayed by a displayed image. Colorfidelity may be degraded due to inaccurate illumination and/or gamma ofa displayed image compared to the original scene. It would be desirableto improve color fidelity for passive displays and display systems.

It would be desirable to improve one or more of the contrast andresolution of and accuracy of image portrayal by passive displays. Italso would be desirable to improve correction of gamma for passivedisplays. It also would be desirable to facilitate such improvements,adjustments and corrections.

A media processor is a device that is used in connection withtelevisions, computer displays, liquid crystal displays, and otherdisplays to receive input signals representing image and/or otherinformation and to provide an output in a format that can be displayed.Sometimes such a media processor is referred to as a media processorintegrated circuit because the circuit and software functions thereofcan be included in a single integrated circuit (or may be in severalintegrated circuits). It would be desirable to include in conjunctionwith a media processor integrated circuit one or more of the otherfeatures described herein.

SUMMARY

An aspect of the invention relates to a system synchronized brightnesscontrol for video images and/or other images that are displayedsequentially using a passive display, including obtaining acharacteristic brightness profile for one or more frames or images in asequence of images and adjusting the illumination intensity (sometimesreferred to as brightness) and/or gamma.

Another aspect of the invention relates to a system synchronizedbrightness control for video images and/or other images that aredisplayed sequentially using a passive display, including a circuitand/or computer software to obtain a characteristic brightness profilefor one or more frames or images in a sequence of images and anadjustment for the illumination intensity (sometimes referred to asbrightness) and/or gamma.

Another aspect relates to using a media processor integrated circuit orthe like capable of receiving different respective input signals andbased thereon providing output signals to operate respective displays,such as, for example, television, HDTV, liquid crystal display, computerdisplay, and/or other display(s) to provide images.

An aspect of the invention relates to a method for displaying an image,including receiving image data representative of respective images,receiving brightness data representative of the brightness of respectiveimages, based on the image data, modulating light from a light source toprovide respective images (for viewing/for projecting), based onbrightness data, adjusting light from the light source to affectbrightness of the image.

Another aspect relates to a storage medium, including a material able toretain data representative of images, image data stored in the material,said image data representative of respective images, brightness datastored in the material, said brightness data representative ofbrightness of respective images, and wherein the storage locations atwhich the brightness data is stored is different from the storagelocations at which the image data is stored.

Another aspect relates to a method of data storage for a sequence ofimages, including storing in a storage medium image data representativeof a sequence of images, storing at a different location in the storagemedium brightness data representative of brightness of respectiveimages.

Another aspect relates to a method of storing image data, includingstoring in a storage medium image information representing respectiveimages, and storing image brightness information in the storage mediumat a separate location from the location at which image information isstored.

Another aspect relates to a system for displaying images and informationusing passive displays as set forth in other claims and summarieshereof, wherein the passive display is a liquid crystal display.

Another aspect relates to a display system for passive displays, whereindata representing illumination characteristics of an input image orscene has been incorporated in a transfer media (e.g., air, modulatedvideo signals, radio signals, cable transmission, storage medium (e.g.,dvd, cd, tape, computer)), including a control responsive to such datato control the optical characteristics of incident light to a passivedisplay to tend to optimize (maximize) contrast or shades of gray in thedisplayed image.

Another aspect relates to a display system for passive displays, whereindata representing illumination characteristics of an input image orscene has been incorporated in a transfer media (e.g., air, modulatedvideo signals, radio signals, cable transmission, storage medium (e.g.,dvd, cd, tape, computer)), including a control responsive to such datato control the optical characteristics of incident light to a passivedisplay to tend to optimize or to maximize color fidelity.

Another aspect relates to a system for preparing data for use indisplaying a sequence of images, including an input to receive imageinformation for use in operating a light modulating display to provide asequence of images, an analyzer to analyze illumination characteristicsof a number of images of such a sequence of images to obtain lightcontrol information for use in controlling optical characteristics ofthe incident light to a light modulating display and gamma so as to tendat least one of to minimize energy usage, to maximize contrast or shadesof gray, and to maximize color fidelity of displayed images.

Another aspect relates to a display system, including a passive display,a source of illumination to illuminate the passive display andcooperative with the passive display to present images, and a transfermedium providing data to control optical characteristics of light fromthe source of illumination and gamma at least one of to minimize energyand to maximize contrast (e.g., shades of gray) in respective displayedimages by the passive display, and wherein the data is based on anevaluation of illumination of an input scene represented by an image fordisplay.

Another aspect relates to a display system for displaying a sequence ofimages, including a passive display, a light source to provide light toilluminate the passive display, drive circuitry to drive the passivedisplay to modulate light from the light source to provide images, and atransfer medium providing image data representing respective images ofan input scene and illumination data representative of an evaluation ofthe illumination of an input scene to control optical characteristics ofthe light source.

Another aspect pertains to for use with a passive display operable to beilluminated by light from a light source to provide images, a transfermedium providing image data representing respective images of an inputscene and illumination data representative of an evaluation of theillumination of an input scene to control optical characteristics ofsuch a light source.

Another aspect pertains to for use with a passive display operable to beilluminated by light from a light source to provide images, a transfermedium providing image data representing respective images of an inputscene and gamma data representative of an evaluation of the illuminationand/or colors of an input scene to control optical characteristics ofsuch a light source.

Another aspect relates to a system for providing image data for displayby an illuminated passive display, including an image obtaining deviceto provide image data representing input scenes, apparatus (e.g.,integrator, averager, weighted averager, standard deviation) to evaluatethe illumination of input scenes to provide illumination data to controlthe optical characteristics of an illumination source for a passivedisplay.

Another aspect relates to a method of editing images, which are composedof an assemblage of pixels (pels, picture elements) for display orprojection using a passive display to which input light is incident toprovide images (e.g., for display or projection), including adjusting acharacteristic of the input light to obtain a desired appearance of thedisplayed image, and storing the adjusted characteristic for usesubsequently to adjust the characteristic of input light to obtain adesired appearance of the image provided by a passive display.

Another aspect relates to a method of reducing the amount of datarequired to provide images from a source to a receiver for display orprojection via a passive display and light source, including separatingintensity data from image data representing an image for display toobtain reduced image data and intensity data, separately providing tothe display and to the light source or light control for the display,respectively, the image data excluding the intensity data for respectiveimages and the intensity data.

Another aspect relates to a method for synchronized brightness controlof video images and/or other images that are displayed sequentiallyusing a passive display, comprising obtaining a characteristicbrightness profile for one or more frames or images in a sequence ofimages, and adjusting at least one of the illumination intensity and/orgamma.

Another aspect relates to a system synchronized brightness control forvideo images and/or other images that are displayed sequentially using apassive display, including a circuit and/or computer software to obtaina characteristic brightness profile for one or more frames or images ina sequence of images and an adjustment for at least one of theillumination intensity and/or gamma.

Another aspect relates to a method of using a media processor integratedcircuit or the like capable of receiving different respective inputsignals and based thereon providing output signals to operate respectivedisplays, such as, for example, television, HDTV, liquid crystaldisplay, computer display, and/or other display(s) to provide images,comprising receiving input data representing image information andbrightness information, and providing such information respectively fordisplaying images and for determining illumination intensity.

Another aspect relates to a method of using a media processor integratedcircuit or the like, including receiving input video signals or the likerepresenting images and brightness of the images, determining brightnessof respective images, and providing a control to control a source ofillumination for a display to display such images at controlledbrightness.

Another aspect relates to obtaining image information pertaining to oneor more scenes, separating image data and illumination data representingthe image information, and providing the image data and illuminationdata to a medium for use in displaying an image using a passive display.

Another aspect of the invention relates to a device that obtains imageinformation pertaining to one or more scenes, apparatus to separateimage data and illumination data representing the image information, anda medium to which the image data and illumination data are provided foruse in displaying an image using a passive display.

An aspect of the invention relates to obtaining image informationpertaining to one or more scenes, separating gamma informationpertaining to the image information and providing image datarepresentative of the image information and the gamma data to a mediumfor use in displaying an image using a passive display.

An aspect of the invention relates to a device that obtains imageinformation pertaining to one or more scenes, apparatus that separatesgamma information pertaining to the image information and a medium towhich image data representative of the image information and the gammadata is provided for use in displaying an image using a passive display.

An aspect of the invention relates to a method for displaying an image,including receiving image data representative of respective images,receiving brightness or illumination level data (as used throughout thispatent application brightness and illumination may be interchangeableunless otherwise indicated by context) representative of the brightnessof respective images, based on the image data, modulating light from alight source to provide respective images for viewing or for projecting,based on brightness or illumination level data, adjusting light from thelight source to affect brightness of the image.

An aspect of the invention relates to a storage medium, including amaterial able to retain data representative of images, image data storedin the material, said image data representative of respective images,brightness data stored in the material, said brightness datarepresentative of brightness of respective images, and wherein thestorage locations at which the brightness data is stored is differentfrom the storage locations at which the image data is stored.

An aspect of the invention relates to a method of data storage for asequence of images, including storing in a storage medium image datarepresentative of a sequence of images, storing at a different locationin the storage medium brightness data representative of brightness ofrespective images.

An aspect of the invention relates to a method of storing image data,including storing in a storage medium image information representingrespective images, and storing image brightness information in thestorage medium at a separate location from the location at which imageinformation is stored.

According to an aspect, a display system for passive displays, whereindata representing illumination characteristics of an input image orscene has been incorporated in a transfer media (e.g., air, modulatedvideo signals, radio signals, cable transmission, storage medium (e.g.,dvd, cd, tape, computer)), and a control responsive to such data tocontrol the optical characteristics of incident light to a passivedisplay to tend to minimize energy usage by a source of such incidentlight.

An aspect of the invention relates to a display system for passivedisplays, wherein data representing illumination characteristics of aninput image or scene has been incorporated in a transfer media (e.g.,air, modulated video signals, radio signals, cable transmission, storagemedium (e.g., dvd, cd, tape, computer)), and a control responsive tosuch data to control the optical characteristics of incident light to apassive display to tend to optimize (maximize) contrast or shades ofgray in the displayed image.

An aspect of the invention relates to a display system for passivedisplays, wherein data representing illumination characteristics of aninput image or scene has been incorporated in a transfer media, e.g.,air, modulated video signals, radio signals, cable transmission, storagemedium, dvd, cd, tape, computer, and a control responsive to such datato control the optical characteristics of incident light to a passivedisplay to tend to optimize or to maximize color fidelity.

An aspect of the invention relates to a system for preparing data foruse in displaying a sequence of images, including an input to receiveimage information for use in operating a light modulating display toprovide a sequence of images, and an analyzer to analyze illuminationcharacteristics of a number of images of such a sequence of images toobtain light control information for use in controlling opticalcharacteristics of the incident light to a light modulating display andgamma so as to tend at least one of to minimize energy usage, tomaximize contrast or shades of gray, and to maximize color fidelity ofdisplayed images.

An aspect of the invention relates to a display system, including apassive display, a source of illumination to illuminate the passivedisplay and cooperative with the passive display to present images, anda transfer medium providing data to control optical characteristics oflight from the source of illumination and gamma at least one of tominimize energy and to maximize contrast (e.g., shades of gray) inrespective displayed images by the passive display, and wherein the datais based on an evaluation of illumination of an input scene representedby an image for display.

An aspect of the invention relates to a display system for displaying asequence of images, including a passive display, a light source toprovide light to illuminate the passive display, drive circuitry todrive the passive display to modulate light from the light source toprovide images, and a transfer medium providing image data representingrespective images of an input scene and illumination data representativeof an evaluation of the illumination of an input scene to controloptical characteristics of the light source.

An aspect of the invention relates to for use with a passive displayoperable to be illuminated by light from a light source to provideimages, a transfer medium providing image data representing respectiveimages of an input scene and illumination data representative of anevaluation of the illumination of an input scene to control opticalcharacteristics of such a light source.

An aspect of the invention relates to a system for providing image datafor display by an illuminated passive display, including an imageobtaining device to provide image data representing input scenes,apparatus (e.g., integrator, averager, weighted averager, standarddeviation) to evaluate the illumination of input scenes to provideillumination data to control the optical characteristics of anillumination source for a passive display.

An aspect of the invention relates to a method of editing images, whichare composed of an assemblage of pixels (pels, picture elements) fordisplay or projection using a passive display to which input light isincident to provide images (for display, as used herein the verb of todisplay, displaying or the like means displaying for direct view,including, but not limited to viewing directly of a display, viewing thedisplay through one or more lenses, reflectors, filters, etc., ordisplaying for projection), including adjusting a characteristic of theinput light to obtain a desired appearance of the displayed image, andstoring the adjusted characteristic for use subsequently to adjust thecharacteristic of input light to obtain a desired appearance of theimage provided by a passive display.

An aspect of the invention relates to a method of reducing the amount ofdata required to provide images from a source to a receiver for display(or projection) via a passive display and light source, includingseparating intensity data from image data representing an image fordisplay to obtain reduced image data and intensity data, separatelyproviding (a) the image data excluding the intensity data and (b) theintensity data to control the passive display and the light source,respectively.

An aspect relates to system including a storage media and an image datafile and illumination data that is in addition to the image data file,the image data file and the illumination data being stored in thestorage media.

An aspect relates to method of editing or adjusting data representingimages of a scene, whether actual or animated, and wherein the data isfor use in a display to portray the scene, comprising adjusting orcorrecting gamma of a number of images represented by the data.

An aspect relates to apparatus for editing information representative ofsequential images representative of a scene, whether an actual scene oran animated scene, comprising an editor, manual or automatic, to provideadjustment of gamma to a number of the sequential images, wherein theadjustments may be different for different respective images.

The present invention can be used in display systems that use displaysthat are directly viewed, those in which there is reflection of lightfrom the displays to provide for viewing of them, those in which lightis projected, e.g., via a lens system or the like, and in virtually anyother type of display system in which a passive display may be used.

Several embodiments and features of the invention are illustrated anddescribed herein. It will be appreciated that the parts, functions,features, etc. of any embodiment may be used in or in combination withanother embodiment.

To the accomplishment of the foregoing and related ends, the inventioncomprises the features hereinafter fully described and particularlypointed out in the claims. The following description and the annexeddrawings set forth in detail certain illustrative embodiments of theinvention. These embodiments, however, are merely indicative of a few ofthe various ways in which the principles of the invention may beemployed. Other objects, advantages and novel features of the inventionwill become apparent from the following detailed description of theinvention when considered in conjunction with the drawings.

Although the invention is shown and described with respect to one ormore embodiments, it is obvious that equivalents and modifications willoccur to others skilled in the art upon the reading and understanding ofthe specification. The present invention includes all such equivalentsand modifications, and is limited only by the scope of the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

In the annexed drawings:

FIG. 1 is a schematic block diagram illustration of an imaging anddisplay apparatus according to an embodiment of the invention;

FIGS. 2A and 2B are schematic block diagram illustrations of imageobtaining systems of the imaging and display apparatus of FIG. 1;

FIGS. 3A and 3B illustrate exemplary data arrangements and storageapproaches according to an embodiment of the invention, wherein FIG. 3Arepresents an exemplary frame and the multiplexing of image data withillumination data and gamma data, and

FIG. 3B represents the arrangement or format of that data;

FIGS. 4-7 are schematic block diagram illustrations of embodiments ofdisplay systems;

FIG. 8 is a schematic block diagram of a signal processing device withan editor feature;

FIG. 9 is a flow chart of an exemplary method for editing data;

FIG. 10 is an illustration of respective sequential pixels for use indescribing embodiments for determining the illumination level of anincident light source for a passive display; and

FIG. 11 is a schematic block diagram of a display system embodying somefeatures of the present invention.

DESCRIPTION

Referring to the drawings, wherein like reference numerals designatelike parts in the several figures, and initially to FIG. 1, an imagingand display apparatus 10 is shown schematically. As is described furtherbelow, the imaging and display apparatus 10 is for passive displays andis operative to evaluate the illumination of an input scene andincorporates data representative of such input scene within a transfermedia. The data controls the optical characteristics of the illuminationsource for the passive display and the gamma in order to tend tominimize energy requirements, to tend to maximize contrast or shades ofgray in the displayed image, and to tend to optimize light sourceoperation for color fidelity. The data provided the transfer media maybe a video signal, modulated video signal, s-video signal, digitalsignal, or any other signal that can be stored in a medium and/or can bebroadcast or otherwise transmitted to a system for displaying images.

The imaging and display apparatus 10 includes an image obtaining system11 and a display system 12. The imaging and display apparatus 10 usesthe image obtaining system 11 to obtain information 13 concerning ascene 14, and the information appropriately is provided to the displaysystem 12, the display system 12 is able to display an imagerepresentative of the scene 14. The output from the display system 12 isan image 15 that can be directly viewed by a viewer, e.g., a person,that is schematically represented at 16. Direct viewing may includedirectly looking at the image 15 or looking at the image via one or morelenses, reflectors, optical filters, etc. The image 15 also may beprojected onto a screen or onto some other surface, which isschematically represented at 17, and the images projected to thatsurface may be viewed by the viewer 16; such projection may be carriedout using various lenses, reflectors, etc.

A dashed line 18 represents a relationship between the image obtainingsystem 11 and the display system 12. That relationship may be a directelectrical connection, an optical signal connection, a broadcastingantenna and receiving antenna system, wireless, wi-fi, a physicalmedium, such as a video tape, DVD, hard drive (magnetic or otherwise)digital memory, solid state memory, or any other mechanism to providedata that represents characteristics of the input information 13 fromthe scene 14 to the image obtaining system 11 so that the display system12 can display images 15 portraying the scene. Collectively therelationship 18 will be referred to below as “medium” that includes allof the foregoing and, thus, may be, for example, an actual broadcastsignal, a physical medium on which data is stored, electrical signal,optical signal, etc. or other connection between the image obtainingsystem 11 and the display system 12, etc.

As is described in further detail below, the image obtaining system 11provides to the medium 18 data as a representation of the (scene) inputinformation 13, and such data is provided as both image data 18 m andillumination data 18 n. The image data represents the various featuresor characteristics of a scene 14, objects in the scene, such as an imageof a tree standing in a field, etc. and the illumination data representsthe illumination of or brightness of the scene 14. As will be describedin further detail below, such image data and/or illumination data may beedited, adjusted, altered, etc. The image data 18 m and illuminationdata 18 n is provided by the signal processing device 21 to the medium18 where it is stored and/or broadcast, e.g., as a television signal,digital signal, or the like. The data 24 (FIGS. 2A and 2B) may be usedin the display system 12 to provide images 15.

Turning to FIG. 2A, the image obtaining system 11 is illustratedschematically. The image obtaining system 11 includes a camera 20 and asignal processing device, circuit or system 21 (referred to as “device”below). The camera 20 may be used to obtain information 13 representingthe scene 14, as in a conventional video camera, digital camera, etc.The camera may be an electrical/electronic type camera that obtains thescene information 13 as an input and provides as an output 23 electronicsignals, e.g., video signals, digital signals, or some other type ofsignals, representing an image of the scene 14. The camera output 23 isprovided as an input to the signal processing device 21. The camera 20may be a film camera that transfers the scene information 13 to film;the film may be developed and scanned or otherwise converted toelectrical, optical or other signals that can be provided in effect asan output 23 and, thus, as an input to the signal processing device 21.The camera 20 may be any other device that obtains input informationregarding a scene and provides representative output signals at 23 as aninput to signal processing device 21. The signals at 23 provided asinput to the signal processing device 21 may be of any of existingformats or formats to be developed in the future, and such signals andformats should be compatible with the capabilities and operation of thesignal processing device 21.

A number of approaches may be used to obtain the image data 18 m and theillumination data 18 n, which are collectively represented at 24. As oneexample, the signals provided by the camera 20 as input 23 to the signalprocessing device 21 may be electronic representations of an image of ascene 14. Those signals may be provided the signal processing device 21in a format allowing the signals to be evaluated to obtain brightness orillumination level of the scene 14.

The output data 24 includes both image data 18 m and illumination data18 n. The image data represents, for example, for a passive liquidcrystal display, DMD, etc., an indication of which pixels are “on”(e.g., providing a light output by reflection or transmission) and whichpixels are “off” (not providing a light output). An image would beprovided by a display based on the on and off pixels and illuminationthereof.

The illumination data represents the illumination level or brightness ofthe scene 14 as input to the image obtaining system 11. The illuminationor brightness can be determined from the input data 23 from the camera20 to the signal processing device 21. For example, if the datarepresenting each respective pixel of information as detected by thecamera 20 includes R, G, B values, and intensity value(s), or includesgray scale information, which includes illumination value, thatinformation can be used by the signal processing device 21 to representbrightness or illumination level of the image that would be formed bythe pixels of a passive display.

To obtain the illumination information for use in displaying respectiveimages by the display system 12, for example, the signal processingdevice 21 may use any of a number of techniques. One exemplary techniqueis to average the illumination data for a given image (or frame), e.g.,by summing the illumination level for each pixel of the frame anddividing by the total number of pixels. Another technique is tointegrate the illumination values of all the pixels of a given frame.Still another technique is to obtain a weighted average or integration,e.g., as by using a standard deviation technique, an average between apredetermined maximum illumination value and minimum illumination value,other than zero, of selected blocks or areas of the image, or some otherapproach to obtain the weighted illumination level. The illuminationinformation can be obtained by looking at, integrating over, averagingfrom all pixel data contained in a particular image or framerepresentation of the image that is to be displayed; or the illuminationinformation may be obtained from less than the entire image or frame,e.g., from nine (or some other number) different locations in the imageor frame; or both.

The signal processing device 21 combines the illumination data and theimage data and provides the same as the output data 24 to the medium 18.The illumination data may be multiplexed with the image data.

Briefly, turning to FIG. 2B, an image obtaining system 11′ (primedreference numerals herein designate parts similar to, but notnecessarily identical to parts that are designated by the same unprimedreference numeral in other drawing figures) is shown. A light meter 25in the image obtaining system 11′ measures the illumination of the scene14, as is represented by the input information line 13′ to the lightmeter 25. The light meter 25 may be any light meter capable of measuringillumination of the scene. The light meter 25 may be able to receivelight from the entire scene 14 or only from a portion of the scene 14,e.g., as with a spot light meter or both. For example, as is illustratedin FIG. 2A, the full scene 14 may be viewed and measured by the lightmeter 25, as is represented by the input line 13′; or the light metermay view and measure light from only a portion, e.g., portion 14 a, ofthe scene 14, as is represented by the dotted line input 13″ to thelight meter. Other possibilities also may be employed, such as, forexample, viewing and measuring several portions (but not all) of thescene, all of the scene and several portions of the scene, all of thescene in respective portions, etc. by the light meter 25. Variousalgorithms may be designed to average, to integrate, to weight, etc.,the results of the light meter measurements to obtain desiredcharacteristics of illumination level data. Similarly, as was describedabove with respect to FIG. 2A, the illumination data may be providedfrom portions (not all of a scene), all of the scene and one or moreportions of the scene, portions (that sum to all of the scene), etc. andthat illumination level data may be weighted, averaged, integrated,summed, etc. as is described herein. The light meter 25 may beadjustable to determine whether the input 13′ is illumination from theentire scene 14 or from only a portion of that scene or both. Havingsuch illumination level data available would provide a movie director,film editor, etc., choices in editing to facilitate various emphasis,highlighting, lighting conditions, and other functions that may bedesired for accurately portraying a scene, for artistic creation, etc.Examples of editing as a feature of the present invention are describedbelow.

Continuing to refer to FIG. 2B, the light meter 25 provides illuminationdata as an input 23′ to the signal processing device 21. The input 23′represents an electrical representation or some other signalrepresentation of the illumination level measured by the light meter 25.The signal processing device 21 may multiplex the illumination leveldata with image data and may provide the same to the medium 18, as wasdescribed above with respect to FIG. 2A. Therefore, the image obtainingsystem 11′ would not necessarily require the signal processing device 21to obtain illumination values from image data received from the camera20, as illumination values are provided from the light meter 25.However, if desired, the signal processing device 21 may carry outfunctions to obtain illumination values from the data representing thepixels from the camera 20 and may effect adjustment of illuminationvalues, comparison of illumination values, etc., based on data from thecamera 20 provided on the input 23 to the signal processing device 21and data provided from the light meter 25 via the input 23′ to thesignal processing device 21. Different light meters 25 may havedifferent light measuring and/or output characteristics that may affectgamma or other parameters; to facilitate compensating for suchvariations, details of the light meter 25 may be provided, e.g., forrecording or for inclusion in a broadcast signal, or may be used in thesignal processing device 21 where that information would be available toprovide suitable compensation in the light source intensity or otherparameters associated with a passive display.

The medium 18 may be a signal storage medium capable of storing imagedata and illumination data 18 n in a desired format, e.g., existingformats or a format that may be developed in the future. In oneembodiment the illumination data is multiplexed with the image data orboth data may be provided in an appropriate way so that in a displaysystem 12 the image data and illumination data can be discerned and usedto provide images 15. The medium 18 may be an optical storage medium,magnetic storage medium, DVD, CD, hard drive, electronic memory, tape,or virtually any device capable of storing the image data andillumination data. As one example, if the medium 18 were a DVD, then theDVD would be able to be provided to a display system 12 that would beable to display images represented by the data stored in the DVD. Themedium 18 may represent broadcasting, for example, the medium 18 may bea broadcast signal that is not necessarily stored in a memory but ratheris broadcast from a transmitter to a receiver. The broadcaster may be atelevision antenna and associated electronics, an electrical, optical orother cable capable of carrying the image data and illumination data toa display system 12 for displaying images or for storage at the displaysystem, e.g., in a VCR, DVD, hard drive, such as a Tivo system or someother device for subsequent display, satellite broadcasting system, etc.Thus, the term “medium” includes both the possibility of being a storagedevice that may be conveyed from one location to another or may beconnected to receive, to store, and to provide for displaying the imagedata and illumination data. Also, the medium 18 may represent atransmission medium or capability to transmit the image data andillumination data from the image obtaining system 11 to a display system12.

The medium 18 may include a data storage system, such as, for example, aCD writing or burning system, DVD writing system, a magnetic datawriting system, e.g., a hard drive, magnetic tape and/or tape drive,etc. The signal processing device 21 may provide for storage in themedium 18 or for broadcasting the image data in combination with theillumination data, the illumination data being multiplexed with theimage data or otherwise combined with the image data. Alternatively, ifdesired, the medium 18 may be provided with both the image data andillumination data, on the one hand, and the image data without aseparate storage of illumination data, e.g., conventional image datawith R, G, B, illumination, and possibly other information provided perpixel, on the other hand; and this would allow reading from the mediumrespective data to display images using a conventional display systemhaving a passive display without an adjustable light source or using adisplay system with a passive display and an adjustable light source.

Referring to FIG. 3A, an exemplary frame 30 is illustrated. The frame 30shows a layout or an organization of the R, G, B data representingrespective pixels of input information 13 of a scene 14 as obtained bythe camera 20. Exemplary pixels 31 a, 31 b, . . . , 31 r in respectiverows 32 a, 32 b, . . . , 32 rs are illustrated. The last pixel of imagedata in the last row of frame 30 is indicated as pixel 32 rs. Each ofthe respective pixels 31 contains suitable image information, such as,for example, R, G, B values, intensity saturation, hue and possiblyother information. The camera 20 itself or with additional circuitry mayprovide such information for the pixels of the exemplary frame 30 in aformat able to be used by the signal processing device 21. That formatmay be conventional of the type typically produced by a video camera,digital video camera, digital camera, or some other camera. That formatmay be a format provided by a scanner, which has scanned images from afilm, photograph, or other source of single or sequence of images. Theformat would be of a type able to be provided for display by atelevision, monitor or other display device without or possibly withadditional signal processing prior to being displayed. Alternatively,the camera 20 may provide the information representing the pixels 31 insome other format. The format should allow the signals to be used by thesignal processing device 21.

The image data 33 representing the information associated with therespective pixels 31 of the exemplary frame 30 is provided to the signalprocessing device 21. In the signal processing device 21 illuminationdata, which is represented by arrow 34, may be obtained in the mannerdescribed above, for example, by averaging, integration, or by someother technique. Alternatively, illumination information may be providedthe signal processing device 21 by a light meter 25. In the signalprocessing device 21 the image data 33 and the illumination data 34 iscombined and then is provided as combined data 35 to the medium 18. Inthe embodiment as illustrated in FIG. 3A, combining of the image data 33and illumination 34 is achieved using a multiplexer 36 to multiplex theillumination data 34 with the image data 33. The multiplexer may be aconventional multiplexer, may be electronic circuitry operated by asuitable software program to effect multiplexing of the data, etc. Thus,an exemplary frame 30 (image data) may be provided to the medium 18 incombination with illumination data representing the illumination levelof an illumination source for a passive display that displays in adisplay system 12 an image represented by the exemplary frame.

Alternatively, or in addition to providing illumination data 34 to becombined with image data 33 for a respective frame 30, gamma data 37also may be provided, and in such case the gamma data 37 is combined,e.g., by multiplexing or some other mechanism, with the image data 33 toprovide combined data 35 to be provided the medium 18. The gamma data,which may be gamma correction data or information 37, therefore, may beprovided with respect to each frame 30 or with respect to a number offrames 30. Therefore, since gamma can be adjusted or corrected on aframe by frame basis or on a group of frames by a group of frames basis,or in any event, if desired, more frequently than in conventionaldisplay systems, the accuracy of the portrayal of a scene 14 by an image15 provided by a display system 12 may be enhanced or increased relativeto prior imaging apparatus and display systems.

Gamma information can be personal to the light source that illuminates ascene when the image of the scene is recorded; and/or gamma informationcan be personal to the recording device, e.g., camera 20. This gammainformation can be recorded in the medium 18 and used to adjust thelight source of a passive display and/or other characteristics orparameters of a passive display to try to obtain to portray the inputscene 14 as accurately as possible by the display.

An example of an embodiment for multiplexing illumination data 34 andgamma data 37 with image data 33 is illustrated schematically in FIG.3B. The pixel data for each of pixels 31 a, 31 b, . . . , 31 r for eachrow 32 a, . . . ,32 s of a frame 30 a is presented sequentially and maybe stored in medium 18, broadcast via medium 18, etc. Sequentiallyfollowing the pixel data are two additional data objects or storagelocations 34 and 37, which contain the illumination level data and thegamma data for the frame 30 a. Frame 30 b follows frame 30 a andlikewise contains pixel data followed by illumination data and gammadata for frame 30 b. Subsequent frames similarly may have illuminationdata and gamma data multiplexed with the pixel data. Another example ofstoring the illumination level data and/or gamma data is to store it inthe interstices of the image data, e.g., at respective locations betweenor within respective pixel data. It will be appreciated that othertechniques may be used to combine image data with illumination dataand/or gamma data.

Using the invention, then, it will be appreciated that illumination dataand/or gamma data or gamma correction data can be included in the medium18 without substantially increasing the amount of data representing theimages in the medium. Therefore, such addition of data does notsignificantly increase the bandwidth required for transferring video orother signals from the image obtaining system 11 to the display system12, whether broadcast or provided via cable, via DVD, or via some othermeans. However, by providing such illumination data and gamma data orgamma correction data, to provide adjustment of images as they aredisplayed by a passive display tends to obtain good color fidelity,contrast, accuracy of image reproduction, etc.

FIG. 4 shows additional details of the display system 12. The medium 18provides the image data 33 and the illumination data 34 and/or gammacorrection data 37, as is shown by the input 40, to display circuitry 41of the display system 12. The display circuitry 41 responds in the usualfashion to the image data to operate a passive display 42, such as aliquid crystal display, by providing the image data via a connection 43.The display circuitry 41 also responds to the illumination data 34and/or the gamma correction data 37 and provides via a connection 44 toa light source 45 (illumination source or illuminating source) todetermine the illumination output from the light source. Theillumination provided by the light source is directed, as is representedby the arrow 46 as incident light to the display 42. The display 42provides an image 15 for viewing by a viewer or projection by aprojector, as is represented at 47. The details of the image 15, e.g.,which pixels are on and which are off, are controlled according to theimage data input 43 to the display 42 and the illumination level of thelight source 45 is adjusted or controlled in response to illuminationand/or gamma data 44.

The display circuitry 41 may include appropriate circuitry and computerprogram software to distinguish between the image data signals and theillumination and/or gamma data from the medium 18. The display circuitry41 may directly operate the display 42 or there may be other amplifiersor other display driving circuitry provided between the displaycircuitry 41 and the display 42, depending, for example, on thecharacter of the display 42. The signal at input 44 to the light source45 may be a control signal that controls the intensity, color, colortemperature or other characteristic or parameter of the light producedby the light source 45, which would have a separate power input; thesignal on line 44 may be the actual power signal that operates the lightsource to provide light output at an intensity level that is a functionof the power signal, color, color temperature or other characteristic orparameter.

In operation the display system 12 responds to signals 40 from themedium 18 to provide images 15 that have an illumination level orbrightness according to the intensity of light from the light source 45.For a bright image the display 42 would provide the various details ofthe image by the respective on and off pixels, and the light source 45would provide a relatively high intensity illumination level to thedisplay. For a relatively dark image, the pixels of the display 42 alsomay be operated to provide accurate representation of the features ofthe image, and the light source 45 would provide a relatively lowerlevel of illumination to the display 42, thus providing a dimmer ordarker image 15. The determination of brightness of the light source 45would be made at the image obtaining system 11 so that the displaysystem 12 may be able to provide promptly the desired image withoutitself having to make computations to determine the illumination levelof the light source 45.

Since the light source 45 does not have to be operated at full intensityall the time, but rather is operated at reduced intensity for thedisplay 42 to display relatively dark scenes, for example, energy isconserved. This power saving feature may increase the effectiveoperational life of batteries in a portable computer device or otherportable display device.

As for gamma correction, various algorithms are known for correctinggamma. In the past usually a fixed gamma correction was made for a givendisplay, and the gamma correction would remain in effect whenever thedisplay would be used. Ordinarily adjustment of gamma correction wouldnot have been possible after it had been set; and even if it werepossible to make adjustments to gamma, usually the gamma correctionwould have been set to a given value and would not be continuouslychanged or able to be continuously changed. In an active display, suchas a CRT, a fixed gamma correction is possible because althoughintensity of output light may change, ordinarily the color of thatoutput light would not change. However, in a passive display in whichthe intensity of the illuminating light may change, color effect alsomay change, e.g., due to changes in color temperature of the lightproduced by the light source. Also, as a given light source for apassive display ages, color temperature also may change and, thus,affect gamma and, therefore, require gamma correction.

In the present invention since the light source intensity can beadjusted and is intended to be adjusted during operation of the displaysystem 12, that adjustment of light intensity also can take into accountgamma correction. Such gamma correction may be according to conventionalor new gamma correction algorithms.

Turning briefly to FIG. 5, a portion of a display system 12′ isillustrated, including a light source 45′ and display 42 that cooperateto provide an image 15. The light source 45′ includes three separatelight sources, red, green, and blue light sources, respectively,designated by the letters R, G, and B. The display system 12 of FIG. 5may operate on a field sequential basis or frame sequential basis.Accordingly, the display circuitry 41 operates the display 42 and thelight source 45′ so as to provide three sequential images to compose theimage 15, namely, a red image, a green image, and a blue image. When thered light source R of the light source 45′ is providing output light,the other green and blue light sources are off, and the display 42 isoperated by the display circuitry 41 to provide the red portion of theimage 15; and similarly, the green and blue portions sequentially areprovided using the green and blue light sources G and B. Such operationallows for adjustment of the intensity of the light provided by thethree light sources are R, G and B of the light source 45′, thereby notonly to control intensity or brightness of the image 15 but also thecolor and for each light source a gamma correction or adjustment effect.Thus, the ability for the image 15 accurately to portray the scene 14may be enhanced compared to conventional display systems.

Color images may be provided using the features of the present inventionin combination with various other types of color display systems thatuse passive displays. For example, a single display having red, greenand blue pixels formed by respective red, green and blue filtersassociated with the pixels may be used with a single light source 45. Anexample is illustrated in the display system 12″ in FIG. 6. The medium18 provides the display circuitry 41 with input data, as was describedabove. The display circuitry 41 operates a reflective display 42′ toturn on or off respective reflective pixels collectively shown at 50with respective red, green and blue (R, G, B) filters. The light 46,e.g., white light or light having selected desired wavelengths, from thelight source 45 illuminates the display, and image 15 is formed byreflection. Similarly, the image could be formed by transmission throughthe display 42′ if a transmissive display were used.

In FIG. 7 is illustrated schematically another display system 12′″ thathas respective red, green and blue light/image engine portions 48R, 48G,48B Each light/image engine portion produces the respective red, greenor blue color portions of the image 15 using a light source and passivedisplay, as well as a filter or coloration of the light source to obtaina respective color for each of the respective portions 48R, 48G, 48B.The display circuitry 41 provides image data and illumination datarepresentative of each color portion of the image 15 to respectivelight/image engine portions as shown at 43, 44. A beam splitterarrangement 49 may be used to combine the image portions to provide theimage 15.

The above embodiments of display systems 12 are exemplary, and it willbe appreciated that other types of display systems may be used inaccordance with the various features of the invention as disclosedherein.

In FIG. 8 an editor feature 50 is shown in association with the signalprocessing device 21′. The signal processing device 21′, as in the caseof the embodiments illustrated and described above, receives an input 23and possibly also 23′ (FIGS. 2A and 2B) as an electronic representationof image information 13 of a scene 14. The editing feature 50 includesan editor 51 an input device 52 for operating the editor 51 and amonitor or display 53. Using the editor feature the image data and/orillumination data 33, 34 (FIG. 3) and, if desired, gamma characteristics37 (FIG. 3) can be edited to change the values thereof. In using theeditor feature 50, a person may observe an image represented by thecurrent values of the image data, illumination data, and gamma data byviewing the monitor/display 53, as is represented in the flow chart ormethod 60 in FIG. 9, and using the input 52 the person may makeadjustments to the data to alter the image as desired. Thus, theproducer or editor of a movie or of some other group of images may makedecisions and adjust light intensity and/or gamma and may provide theresulting adjusted/edited movie, images, etc. in or to the medium 18 forstorage, broadcasting, etc. Color effects also can be adjusted/edited,e.g., to create a scene that appears to be at sunset, taken in a candlelit room or cave, under water, etc., by adjusting illumination intensityof the back light in general or of respective color components (R, G, B,for example) of the image; and the resulting edited data can be providedthe medium 18.

As is shown in FIG. 9, a person may view an image at block 61 and makemental decisions or have the editor 51 make the decisions based onconventional image adjusting algorithms, e.g., to adjust brightness,contrast, and/or gamma, or algorithms that may developed in the futureas to the quality of the image, characteristics of the image, etc. Atblock 62 an inquiry is made whether to edit data of such viewed image.If editing is desired, then the data is edited at block 63, and then theimage is viewed again (or is constantly viewed as the data is edited).If no editing is desired or required or if editing has been completed,then from block 62 the flow chart/method goes to block 64 to send thedata to the medium 18 for broadcasting, storage, etc. Such editing canbe carried out for each image, such as for each image frame 30 (FIG. 3)or for each other assemblage of image data representing an image. Suchediting may be carried out for a group of images or frames. For example,if several frames have similar data, then the editing of one of thoseframes may be used to adjust the data for the other related or similarframes.

The editor feature 50 may be constantly available and set to a mode thateach frame or each group of frames is to be reviewed via themonitor/display 53, for example, for editing. The editor feature 50 maybe selectively set (a) to a bypassed condition in the signal processingdevice 21′ so that no editing would be applied to the data that isotherwise automatically processed in the signal processing device 21′ inthe manner described above; or (b) to an activated condition to carryout editing. For example, in the case of a high quality movie, theeditor feature 50 would be activated so that editing is applied.Alternatively, for a relatively low-cost movie it may be desired todeactive the editor feature 50 to save time and cost of production.

The editor feature 50 is shown in FIG. 8 as part of the signalprocessing device 21′. The other portions of the signal processingdevice 21′ are not illustrated but may be circuitry, devices, andoperation, e.g., by software, as was described above. However, it willbe appreciate that the editor feature 50 may be separate from the signalprocessing device. An exemplary device for use as the editor feature 50may be a computer system including a monitor/display 53, input/outputdevices, such as a keyboard, mouse, joy stick, pointer, etc., andappropriate memory and storage devices. Such a computer system may beconnected to the signal processing device to receive data from it fordisplay on the monitor/display 53 and for editing the data for return tothe signal processing device and ultimately to be provided to the medium18. Such a computer system also may be included as part of the signalprocessing device 21′ as is illustrated at 50 in FIG. 8. Such computersystem may use a suitable computer program to carry out the editingfunctions; an example of a suitable computer program is that sold underthe trademark Adobe Photoshop, version 7.0. However, it will beappreciated that other existing computer program software may be usedfor editing.

A number of different techniques may be used for determining theillumination level for the light source 45, 45′ (FIGS. 4, 5) asrepresented by illumination data. Several examples are described above,including obtaining an average, a weighted average, integration, orweighted integration of the brightness information obtained by thecamera 20 or brightness information obtained by a light meter 25 (FIGS.2A , 2B). Such brightness information may be obtained by a single frameor image or from a portion, from portions and/or from all of a frame orimage, and used to determine the illumination level of the light sourcewhen the image represented by that frame is provided by the display 42.

Another method for determining the illumination level is to use theillumination level information from a number of frames to determine theillumination level for a given frame. This method may use illuminationlevel from several frames that have been displayed prior to the givenframe being displayed, from several frames that will be displayedfollowing the displaying of the given frame, or from several framespreceding and following the given frame.

In the editing process various adjustments could be made to enhancecertain characteristics of the displayed image; and the algorithms foradjusting the light source 45, 45′ also may be set to accomplish suchenhancement. In an exemplary embodiment assume a dark scene at nightalong with a bright street light in the image or a momentary lightningflash. By setting the integration function to a desired weight todetermine the illumination of a frame representing such a scene or bycarrying out appropriate editing steps, the bright portion of the imagecould be highlighted or diffused. For example, the intensity of thelightning flash may increase the integrated brightness of the imagesubstantially, which would reduce the difference in intensity between,say, a bolt of lightning, and the balance of the image; in contrast, bydiminishing the impact of the lightning flash, e.g., by maintaining theintensity of the balance of the image at a relatively dark scene level,the lightning bolt would tend to stand out or to be highlighted.

In FIG. 10 a sequence of frames 70 is illustrated schematically. Thesequence of frames includes, among others, frames a-i. For thisembodiment it is assumed that the brightness or illumination level ofthe light source for displaying frame e is to be determined, and it alsois assumed that frames a-d would be displayed sequentially prior to thedisplaying of the image represented by the data of frame e and that theimages represented by the data of frames f-i would be displayedsubsequent to the displaying of the image represented by frame e.

If the illumination level of the given frame e is to be determined basedon the illumination level of preceding frames, a number of thosepreceding frames would be selected, e.g., three frames, meaning in thisexample frames b, c, and d (or one or more other number of frames). Theillumination level of those three frames may be determined and used todetermine the illumination level of frame e. In another example, theillumination level of the three preceding frames (or any numberpreceding frames as are selected or preselected or set in the signalprocessing device 21, 21′ is combined with the illumination level offrame e. Combining of illumination levels may be, for example, obtainingan average of the illumination levels of the respect frames; thoseillumination levels included in the average may be obtained byaveraging, weighted averaging, integration, weighted integration, orsome other method, as may be desired. A similar approach may be used toobtain an illumination level for the given frame e using theillumination levels of one or more subsequent frames alone or incombination with the illumination level of the given frame e.Furthermore, if desired, the illumination level for the given frame emay be obtained using the illumination level information from one ormore preceding frames and one or more subsequent frames.

A number of advantages may be achieved using the method described abovewith respective FIG. 10 to obtain an illumination level for respectivegiven frames using illumination level of one or more other frames. Forexample, in the event there were an aberration in the illumination of agiven frame due to a data error, due to an unintended instantaneous andbrief change in illumination of the scene, e.g., such as may beencountered in an indoor movie sound stage or setting having a givenlight level when a door to the bright ambient sunlit outside is opened,a light switch mistakenly is turned on during filming, etc.; the effectof such brief change in illumination is in a sense averaged out so thatit has relatively little impact and does not require the re-filming of agiven scene. Another advantage is the reducing of the amount of editingrequired for a series of frames and/or the averaging out an error thatmay occur when editing a single frame (or several frames) or themistaken or intentional omitting of editing of a signal frame (or ofseveral frames). Another advantage would be the ability to provide forrelatively smooth transitions in illumination level even as sceneschange for a given movie, e.g., from a relatively dark scene to arelatively bright scene. Sharp changes in illumination level may be eyecatching, on the one hand, but such eye catching effect may bedisadvantageous when one person is watching a movie while another personis trying to sleep—the sharp change in illumination may awaken thesleeper. The softer effect or somewhat gradual change in illuminationlevel also may make the viewing of a movie more comfortable, e.g., lessharsh on the eyes.

The above-described techniques for editing illumination level using oneor more frames different from or in addition to the given frame also maybe used to provide gamma adjustment for the given frame. Also, thenumber of frames and whether the given frame is included in thedetermining of a illumination level or gamma characteristics may beselected by an editor using the editor feature 50 (FIG. 8).

Briefly referring to FIG. 11, a display system 80 receives input signaldata, e.g., a video signal or some other signal representing an image,brightness information, gamma information and/or gamma correctioninformation, etc., for example as is described herein, at an input 81.The input 81 may receive the signals from an antenna, an electrical oroptical cable, a satellite, a CD or DVD player, a tape player, etc. Theinput signal data is provided a media processor 82, such as a mediaprocessor integrated circuit. Included in the media processor integratedcircuit is associated hardware, firmware and/or software to obtainand/or to respond to image data 83 and brightness data 84, and,accordingly, the media processor integrated circuit 82 may provide imagedata via a connection 85 to the display 42 to create respective imagesand brightness data via a connection 86 to a light source 45 toilluminate the display at respective brightness levels.

If desired, the media processor integrated circuit 82 may receive inputsignal data as conventional video signals or other similar signals andmay include a signal processing device 21, e.g., as was described above,to provide image data and brightness data to the display 42 and lightsource 45, respectively in a manner similar to what is described aboveor in other equivalent manner.

INDUSTRIAL APPLICATION

The present invention may be used to obtain, to store and to displayimages.

1-48. (canceled)
 49. A display system for passive displays, wherein thedisplay system is configured to receive an input signal includingseparately encoded display illumination control data and image data, thedisplay system comprising: a control configured to process theseparately encoded display illumination control data included in theinput signal and to control the optical characteristics of incidentlight to a passive display based on the separately encoded displayillumination control data.
 50. The display system of claim 49, whereinthe control is configured to process the display illumination controldata based on a maximized number of shades of gray in the displayedimage.
 51. An image processing system for preparing control data for usein displaying a sequence of images, such control data to be includedwith input image data to a passive display with an illuminating lightsource and used to control the illuminating light source, the imageprocessing system comprising: a buffer memory to receive image datarepresentative of the sequence of images, an analyzer configured toanalyze image data of a plurality of images of the sequence of images,to create illumination control data for use in controlling opticalcharacteristics of the incident light to a light modulating display, andto separately encode the illumination control data together with theinput image data in a combined output signal of the image processingsystem to be used as input to a passive display with an illuminatinglight source.
 52. The image processing system of claim 51, furthercomprising a transfer media for transferring the combined output signalincluding the illumination control data and the image data to a display,said transfer media comprising at least one of broadcast radio signalsor video signals that include the illumination control data with theimage data.
 53. The image processing system of claim 52, said transfermedia comprising a storage medium, said storage medium comprising atleast one of a dvd, cd, tape, computer memory or hard drive.
 54. Thesystem of claim 51, further comprising a multiplexer for timemultiplexing the control data with the image data.
 55. A display system,comprising: a passive display, a source of illumination to illuminatethe passive display and cooperative with the passive display to presentimages, and a storage medium providing an input signal includingseparately encoded illumination control data and image data, theillumination control data controlling optical characteristics of lightfrom the source of illumination, and wherein the illumination controldata is based on an evaluation of illumination of an input scenerepresented by an image for display, the evaluation occurring prior tostorage of the illumination control data on the storage medium.
 56. Thesystem of claim 55, wherein the illumination control data is based on anevaluation of illumination of an input scene represented by severalimages for display, and wherein the passive display is operable todisplay a sequence of images, and wherein the several images include anumber of images in the sequence of images preceding or following agiven image for display.
 57. An image processing system for preparing asignal that includes image data and control data for a passive displaywith an illuminating light source, the image processing systemcomprising: an image obtaining device to provide image data representinginput scenes, an image processing apparatus configured to evaluate imagedata of input scenes, and to provide separately encoded display controldata to be included with the image data in the signal, the separatelyencoded display control data including illumination control data tocontrol the optical characteristics of an illumination source for apassive display.
 58. A method of editing and encoding images representedby image data captured by an image recording device, where the capturedimages are composed of an assemblage of pixels for display or projectionusing a passive display to which input light is incident to provideimages, comprising: determining display control data to be used in thedisplay to obtain a desired appearance of a displayed image includingcontrol data for a characteristic of the input light, including thedetermined display control data with the image data, the display controldata corresponding to the adjusted light characteristic, and storingboth the display control data and the image data for use subsequently toadjust the characteristic of input light to obtain a desired appearanceof the image provided by a passive display with an illuminating lightsource.
 59. The method of claim 58, further comprising visually viewingan image while determining the display control data to adjust acharacteristic of the input light to obtain a desired appearance of thedisplayed image.
 60. The method of claim 58, said adjusting comprisingmanually adjusting.
 61. The method of claim 58, further comprisingobtaining the illumination of a captured scene portrayed by an image,and including display control data to provide an image havingillumination that is substantially the same as the captured scene. 62.The method of claim 61, said obtaining comprising at least one ofobtaining an average value of the illumination of the captured scene,computing the illumination of the captured scene by obtaining a spatialdistribution of light intensity over the area of the input scene, andobtaining illumination information from a number of images of respectivescenes.
 63. The method of claim 62, wherein the image is a given imagein a sequence of images assembled to provide a motion picture effect,and wherein said obtaining illumination information from a number ofimages of respective scenes comprises obtaining such illuminationinformation from a number of images in the sequence that precede orfollow the given image in the sequence.
 64. A display system comprising:a passive display; an illumination source to illuminate the passivedisplay and cooperate with the passive display to display images; anddisplay circuitry operatively coupled to the passive display and theillumination source and configured to receive an input signal whereinthe input signal includes separately encoded image data and illuminationcontrol data; and wherein the display circuitry is configured to processthe separately encoded illumination control data included in the inputsignal, and to control the level of the illumination source based on theseparately encoded illumination control data.
 65. The display system ofclaim 64, wherein the illumination source includes a red light source, agreen light source and a blue light source, and the illumination controldata includes color control data for controlling the source ofillumination based on the color control data.
 66. A display systemcomprising: a passive display; an illumination source to illuminate thepassive display and cooperate with the passive display to displayimages; and display circuitry operatively coupled to the passive displayand the illumination source and configured to receive an input videosignal indicative of an image to be displayed, wherein the input videosignal includes separately encoded image data and display illuminationcontrol data; and wherein the display circuitry includes decodingcircuitry configured to decode and process the separately encodeddisplay illumination control data within the input video signal andcontrol the passive display and the illumination source based on theseparately encoded display illumination control data.
 67. A method ofproviding images to a passive display for display by the passivedisplay, the method comprising: storing separately encoded (i) imagedata representative of respective images and (ii) display control data,the display control data comprising illumination control data for use incontrolling optical characteristics of incident light to the passivedisplay and image control data for controlling processing of the imagedata by the passive display to control characteristics of displayedimages; and transmitting an input signal to the passive display, theinput signal including the separately encoded image data and displaycontrol data.
 68. The method of claim 67, wherein the image control datais configured to maximize shades of gray of displayed images.
 69. Themethod of claim 67, wherein the image control data is configured tocontrol gamma of the passive display.
 70. The method of claim 67,wherein the image control data is configured to modify gamma of thepassive display.